PDQ ATS research system

  • Users new to R
  • TAQ data assembled into partial books
  • Running in Amazon EC2
  • Research focused on transaction costs




Support diverse exploratory research programs




Running across all tickers from days to months or so




Lightweight software footprint, easy to use



You know how this starts

  • Some data, maybe a lot of data
  • Maybe a database
  • Complicated modeling and research; need R or Python to be productive
  • Then, maybe throw even more analysis tools into the mix
  • You find yourself adding more computing horsepower
  • Before you know it, you end up with…




Eventually, some of those arrows connect to R

Yuck!


SciDB streaming API




Databases a great at managing mutable data

Append-only data are much simpler to work with



Elastic computing with R and Redis on Amazon EC2



Amazon EC2 Recipe

  1. Instantiate a Linux image
  2. Install R and desired packages, including doRedis
  3. Run the doRedis service installer script
  4. Make an AMI from this instance
  5. Re-launch from AMI as needed, supplying config as EC2 user data